Chapter 7 Inferential Statistics (Concepts)

Population and Sample

---

In the field of statistics, we are often interested in understanding the characteristics or properties of a large group of individuals, objects, or data points. For example, we might want to know the average income of all households in a city, the effectiveness of a new medicine on all patients with a certain disease, or the quality of all products produced by a factory. This entire group that we are interested in is called the Population.

A Population is the complete set of all possible observations, individuals, units, or data that share a common characteristic of interest for a particular study. Defining the population is the first crucial step in any statistical investigation, as it clearly specifies the group about which conclusions are to be drawn. The size of a population is denoted by $N$.

Examples of populations:

• All adult citizens of India eligible to vote in a general election.
• All students currently enrolled in Class 12 in a specific state board.
• Every product manufactured by a specific company within a defined period.
• All the fish of a particular species in a specific lake.
• All possible outcomes of repeatedly rolling a standard six-sided die (an infinite population).

While our goal is to understand the population, collecting data from every single member of the population (conducting a census) is often impractical, resource-intensive, or even impossible, especially if the population is very large, geographically dispersed, or constantly changing.

---

Sample

Since studying the entire population is rarely feasible, statisticians resort to studying a smaller, more manageable subset drawn from the population. This subset is called a Sample.

A Sample is a subgroup or subset of the population from which data is actually collected. The data collected from the sample is then used to make generalizations or inferences about the characteristics of the entire population. The size of the sample is denoted by $n$, where typically $n < N$.

Examples of samples corresponding to the populations mentioned above:

• A randomly selected group of 2500 adult citizens interviewed for an opinion poll.
• A selection of 200 students from different schools following that state board.
• A batch of 50 products randomly picked from the production line for quality testing.
• A net cast in different locations in the lake to catch a number of fish of that species.
• The results recorded from rolling the die 100 times.

The key to making valid inferences about the population from a sample is ensuring that the sample is representative of the population. A representative sample accurately reflects the characteristics of the population from which it was drawn. The best way to obtain a representative sample is typically through random sampling methods, where each member of the population has a known and non-zero chance of being selected for the sample. This helps in avoiding selection bias.

---

Why Study a Sample Instead of the Population?

There are several compelling reasons why sampling is almost always preferred over conducting a full census of the population:

• Cost-Effectiveness: Collecting data from a sample is significantly less expensive in terms of manpower, materials, and travel compared to a census.
• Time-Efficiency: Data can be collected, processed, and analysed much faster from a sample, allowing for timely results and decision-making.
• Feasibility and Practicality: For large or geographically dispersed populations, conducting a census might be logistically impossible. Some populations are hypothetical or infinite (like the outcomes of a process), making sampling the only option.
• Destructive Testing: If the process of collecting data involves destroying the item being tested (e.g., testing the lifespan of a battery, testing the strength of a material), then sampling is essential as a census would destroy the entire population.
• Accuracy: Sometimes, a carefully conducted sample study can even yield more accurate results than a large census. This is because smaller-scale studies allow for better training of personnel, closer supervision, and more careful data collection and processing, potentially reducing non-sampling errors.

The goal of Inferential Statistics, which is the subject of this chapter, is precisely to develop methods for using information gathered from a sample to draw reliable conclusions or make inferences about the unknown characteristics of the entire population.

Parameter, Statistic, and Statistical Inference

---

When we conduct a statistical study, whether we are examining a population or a sample, we are typically interested in measuring certain numerical characteristics that describe the group. These characteristics can be measures of central tendency (like averages), measures of dispersion (like variability), or measures of proportion (like percentages). The terminology used to describe these characteristics depends on whether they pertain to the entire population or just to a sample drawn from that population. This distinction is fundamental to understanding inferential statistics.

---

Parameter

A parameter is a numerical characteristic that describes some aspect of the entire population. Parameters are considered fixed values for a given population, although in most real-world scenarios, their true values are unknown to us because we rarely have data for the entire population.

Parameters are typically denoted by Greek letters. Some common population parameters include:

• Population Mean: The average value of a variable for all individuals in the population, denoted by $\mu$ (mu).
• Population Standard Deviation: A measure of the spread or variability of the variable's values across the entire population, denoted by $\sigma$ (sigma).
• Population Variance: The square of the population standard deviation, denoted by $\sigma^2$.
• Population Proportion: The proportion or percentage of individuals in the population that possess a certain characteristic, often denoted by $p$ or $\pi$ (pi).
• Population Median, Mode, etc.: Other measures of central tendency or distribution spread for the population.

For example, if we are interested in the average height of all adult males in Delhi, the true average height of all adult males in Delhi is the population mean ($\mu$). If we are interested in the percentage of voters in Karnataka who support a particular political party, the actual percentage among all voters in Karnataka is the population proportion ($p$). These values exist, but we usually don't know them exactly.

---

Statistic

A statistic (or sample statistic) is a numerical characteristic that describes some aspect of a sample. Statistics are calculated directly from the data collected from the sample. Unlike parameters, the value of a statistic is known once the sample data is collected.

Since a sample is only a part of the population, and different samples drawn from the same population are likely to contain different individuals, the value of a statistic can vary from sample to sample. This variability of statistics is a key concept in inferential statistics.

Statistics are typically denoted by Roman letters. Some common sample statistics include:

• Sample Mean: The average value of a variable for the individuals in the sample, denoted by $\bar{x}$ (x-bar). It is calculated as $\bar{x} = \frac{\sum x_i}{n}$, where $x_i$ are the values in the sample and $n$ is the sample size.
• Sample Standard Deviation: A measure of the spread or variability of the variable's values within the sample, denoted by $s$. It is calculated using a specific formula (often with $n-1$ in the denominator for better estimation of $\sigma$).
• Sample Variance: The square of the sample standard deviation, denoted by $s^2$.
• Sample Proportion: The proportion or percentage of individuals in the sample that possess a certain characteristic, often denoted by $\hat{p}$ (p-hat). It is calculated as $\hat{p} = \frac{\text{Number of successes in the sample}}{\text{Sample size}} = \frac{x}{n}$.
• Sample Median, Mode, etc.: Other measures calculated from the sample data.

For instance, if we take a sample of 50 adult males from Delhi and calculate their average height, this calculated average is the sample mean ($\bar{x}$). This sample mean is used as an estimate for the unknown population mean height ($\mu$). Similarly, if we survey 500 voters in Karnataka and find that 45% of them support a particular party, then 45% is the sample proportion ($\hat{p}$), which serves as an estimate for the unknown population proportion ($p$).

The table below summarizes the distinction:

Characteristic	Population (Parameter)	Sample (Statistic)
Refers to	The entire group of interest	A subset of the population
Value	Fixed, but typically unknown	Varies from sample to sample, is calculated from data
Notation (Examples)	$\mu$ (mean), $\sigma$ (standard deviation), $\sigma^2$ (variance), $p$ or $\pi$ (proportion)	$\bar{x}$ (mean), $s$ (standard deviation), $s^2$ (variance), $\hat{p}$ (proportion)
Goal	To describe the population	To describe the sample and make inferences about the population

---

Statistical Inference

Statistical inference is the main objective when we collect data from a sample. It is the process of using the information obtained from a sample to draw conclusions, make estimations, or test hypotheses about the characteristics (parameters) of the larger population from which the sample was drawn. Since we are using partial information (from the sample) to make statements about the whole (the population), statistical inference always involves some degree of uncertainty. A key part of statistical inference is quantifying this uncertainty, usually using probability.

The two primary branches of statistical inference are:

Estimation

Estimation involves using sample statistics to estimate the unknown value of population parameters. There are two main types of estimation:

• Point Estimation: This involves calculating a single value (a point estimate) from the sample data that is considered the "best guess" for the population parameter. For instance, the sample mean $\bar{x}$ is a point estimate for the population mean $\mu$. The sample proportion $\hat{p}$ is a point estimate for the population proportion $p$.
• Interval Estimation (Confidence Intervals): This involves constructing a range of values (an interval estimate) within which the population parameter is likely to lie, with a certain level of confidence. A confidence interval provides more information than a point estimate alone because it includes a measure of the precision or reliability of the estimate. For example, instead of just stating that the estimated average height is 162 cm (point estimate), we might say that we are 95% confident that the true average height of the population is between 160 cm and 164 cm (interval estimate).

Hypothesis Testing

Hypothesis testing (also known as significance testing) is a formal procedure used to evaluate a claim or statement (a hypothesis) about a population parameter using evidence from sample data. It involves a structured process:

1. Formulate Hypotheses: State a null hypothesis ($H_0$), which is typically a statement of no effect or no difference (e.g., the average income is ₹50,000). State an alternative hypothesis ($H_1$ or $H_a$), which is contrary to the null hypothesis (e.g., the average income is not ₹50,000, or is greater than ₹50,000).
2. Select Significance Level: Choose a level of significance ($\alpha$), which is the probability of rejecting the null hypothesis when it is actually true (Type I error). Common values are $\alpha = 0.05$ (5%) or $\alpha = 0.01$ (1%).
3. Collect Sample Data: Obtain a random sample from the population.
4. Calculate Test Statistic: Compute a value from the sample data (a test statistic, e.g., a t-statistic or z-statistic) that measures how far the sample result deviates from what is expected under the null hypothesis.
5. Determine P-value or Critical Value: Compare the test statistic to a known probability distribution (like the t-distribution or standard normal distribution) to find the p-value (the probability of observing a sample result as extreme as, or more extreme than, the one obtained, assuming the null hypothesis is true) or compare the test statistic to critical value(s) from the distribution based on the chosen $\alpha$.
6. Make a Decision: If the p-value is less than $\alpha$ (or if the test statistic falls into the rejection region defined by the critical value(s)), we reject the null hypothesis. Otherwise, we fail to reject the null hypothesis.
7. Draw Conclusion: State the conclusion in the context of the original problem, interpreting what the decision means regarding the population parameter.

Hypothesis testing allows us to make probabilistic statements about the strength of evidence against the null hypothesis. For example, based on sample data, we might conclude that there is sufficient evidence to reject the claim that the average income is ₹50,000 and infer that it is likely different.

Inferential statistics provides the tools and methods to bridge the gap between sample information and population knowledge, enabling data-driven decision-making in situations where complete population data is unavailable.

t-Test (One Sample and Two Independent Groups)

---

In inferential statistics, one of the most frequent tasks is comparing means. We might want to compare the mean of a single sample to a known or hypothesized value, or compare the means of two different groups. The t-test is a powerful and widely used statistical method for performing such comparisons, particularly when dealing with sample data and when the population standard deviation is unknown.

The t-test is based on the t-distribution (also known as Student's t-distribution), which is similar in shape to the normal distribution but has heavier tails. This means it has more probability in the tails and less in the center compared to the normal distribution. The shape of the t-distribution depends on a parameter called the degrees of freedom (df), which is related to the sample size. As the sample size (and thus the degrees of freedom) increases, the t-distribution approaches the standard normal distribution.

---

Assumptions of the t-Test

For the results of a t-test to be valid, certain assumptions about the data should ideally be met. While the t-test is relatively robust to minor violations of some assumptions, especially with larger sample sizes, it's important to be aware of them:

• Independence of Observations: The data points within each group should be independent of each other. If comparing two groups, the data points in one group should be independent of the data points in the other group. This is typically ensured through proper study design and random sampling/assignment.
• Approximate Normality: The data for the variable being tested should be approximately normally distributed in the population(s) from which the sample(s) were drawn. However, the t-test is quite robust to violations of normality, especially when the sample size(s) are large (e.g., $n > 30$ per group), thanks to the Central Limit Theorem, which ensures that the sampling distribution of the mean is approximately normal even if the population distribution is not.
• Random Sampling: The sample(s) should be drawn from the population(s) using random sampling methods. This helps ensure that the sample is representative of the population and supports the assumption of independence.

For the two-independent-samples t-test (especially the pooled variance version), an additional assumption is:

• Homogeneity of Variances (Equal Variances): The variances of the populations from which the two independent samples are drawn are equal ($\sigma_1^2 = \sigma_2^2$). If this assumption is not met, an alternative version of the t-test (Welch's t-test) should be used.

---

One-Sample t-Test

The one-sample t-test is used to determine if the mean of a single sample is statistically different from a known or hypothesized value for the population mean. This hypothesized value is often based on prior research, a theoretical expectation, or a standard.

Purpose

To test whether the sample mean ($\bar{x}$) provides sufficient evidence to conclude that the true population mean ($\mu$) is different from a specific value ($\mu_0$).

Hypotheses

We formulate a null hypothesis ($H_0$) and an alternative hypothesis ($H_1$ or $H_a$) about the population mean ($\mu$) in relation to the hypothesized value ($\mu_0$).

• Null Hypothesis ($H_0$): States that there is no significant difference between the true population mean and the hypothesized value.

  $H_0: \mu = \mu_0$

  [The true population mean is equal to the hypothesized value]

• Alternative Hypothesis ($H_1$): States that there is a significant difference. This can be:
  • Two-sided: The population mean is not equal to the hypothesized value.

    $H_1: \mu \ne \mu_0$

  • One-sided (Right-tailed): The population mean is greater than the hypothesized value.

    $H_1: \mu > \mu_0$

  • One-sided (Left-tailed): The population mean is less than the hypothesized value.

    $H_1: \mu < \mu_0$

Test Statistic

The one-sample t-statistic measures how many standard errors the sample mean ($\bar{x}$) is away from the hypothesized population mean ($\mu_0$). When the population standard deviation ($\sigma$) is unknown (which is typically the case when using a t-test), we use the sample standard deviation ($s$) to estimate the standard error of the mean. The standard error of the mean is $\frac{\sigma}{\sqrt{n}}$, and its estimate is $\frac{s}{\sqrt{n}}$.

The formula for the one-sample t-statistic is:

where:

• $\bar{x}$ is the sample mean calculated from the collected data.
• $\mu_0$ is the hypothesized population mean (the value from the null hypothesis $H_0$).
• $s$ is the sample standard deviation.
• $n$ is the sample size.
• $\frac{s}{\sqrt{n}}$ is the estimated standard error of the mean.

Under the null hypothesis, this t-statistic follows a t-distribution with degrees of freedom $df = n - 1$. The degrees of freedom represent the number of independent pieces of information available to estimate the population variance.

Decision Rule

To make a decision about whether to reject $H_0$, we compare the calculated t-statistic to critical values from the t-distribution table or use the p-value approach. This comparison is done at a pre-determined significance level, $\alpha$.

• Using Critical Values:

  For a given significance level $\alpha$ and degrees of freedom $df = n-1$, we find the critical value(s) $t_{critical}$ from the t-distribution table.

  • For a two-tailed test ($H_1: \mu \ne \mu_0$), there are two critical values, $\pm t_{\alpha/2, df}$. We reject $H_0$ if $|t_{calculated}| > t_{\alpha/2, df}$.
  • For a right-tailed test ($H_1: \mu > \mu_0$), there is one critical value, $t_{\alpha, df}$. We reject $H_0$ if $t_{calculated} > t_{\alpha, df}$.
  • For a left-tailed test ($H_1: \mu < \mu_0$), there is one critical value, $-t_{\alpha, df}$. We reject $H_0$ if $t_{calculated} < -t_{\alpha, df}$.
• Using P-value:

  The p-value is the probability of obtaining a test statistic as extreme as, or more extreme than, the one calculated from the sample, assuming the null hypothesis $H_0$ is true. A smaller p-value indicates stronger evidence against $H_0$.

  • We calculate the p-value based on the $t_{calculated}$ and $df$.
  • We compare the p-value to the significance level $\alpha$.
  • If p-value $\le \alpha$, we reject $H_0$.
  • If p-value $> \alpha$, we fail to reject $H_0$.

Rejecting $H_0$ suggests that there is statistically significant evidence, based on the sample, to support the alternative hypothesis. Failing to reject $H_0$ means the sample data does not provide enough evidence to contradict the null hypothesis; it does *not* mean that $H_0$ is true.

---

Two Independent Samples t-Test

The two independent samples t-test (also called the unpaired t-test or independent groups t-test) is used to determine if there is a statistically significant difference between the means of two independent groups. "Independent" means that the individuals or observations in one group are unrelated to the individuals or observations in the other group (e.g., comparing the average height of males and females, where individuals are measured once and belong to only one group).

Purpose

To test whether the difference between the sample means of two independent groups ($\bar{x}_1$ and $\bar{x}_2$) is large enough to conclude that the true population means ($\mu_1$ and $\mu_2$) of the two groups are different.

Hypotheses

Let $\mu_1$ and $\mu_2$ be the true population means for the two groups.

• Null Hypothesis ($H_0$): States that there is no significant difference between the population means of the two groups.

  $H_0: \mu_1 = \mu_2$

  [Equivalent to $H_0: \mu_1 - \mu_2 = 0$]

• Alternative Hypothesis ($H_1$): States that there is a significant difference. This can be:
  • Two-sided: The population means are not equal.

    $H_1: \mu_1 \ne \mu_2$

  • One-sided (Right-tailed): The mean of group 1 is greater than the mean of group 2.

    $H_1: \mu_1 > \mu_2$

  • One-sided (Left-tailed): The mean of group 1 is less than the mean of group 2.

    $H_1: \mu_1 < \mu_2$

Test Statistic (Assuming Equal Variances - Pooled t-Test)

This version of the two-independent-samples t-test assumes that the population variances of the two groups are equal ($\sigma_1^2 = \sigma_2^2$). If this assumption is reasonable or justified, pooling the sample variances provides a better estimate of the common population variance.

First, we calculate the pooled sample variance ($s_p^2$):

where:

• $n_1$ and $n_2$ are the sample sizes of Group 1 and Group 2, respectively.
• $s_1^2$ and $s_2^2$ are the sample variances of Group 1 and Group 2, respectively.

The term $(n_1-1)$ and $(n_2-1)$ are the degrees of freedom for each sample variance. The denominator $n_1 + n_2 - 2$ is the total degrees of freedom for the pooled variance estimate.

The two-sample t-statistic measures the difference between the two sample means relative to the estimated standard error of the difference between means. The estimated standard error of the difference between means (when variances are pooled) is $\sqrt{s_p^2 \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}$.

The formula for the pooled two-independent-samples t-statistic is:

Under the null hypothesis $H_0: \mu_1 = \mu_2$, which means $\mu_1 - \mu_2 = 0$, the formula simplifies to:

where:

• $\bar{x}_1$ and $\bar{x}_2$ are the sample means of Group 1 and Group 2.
• $s_p^2$ is the pooled sample variance calculated using formula (2).
• $n_1$ and $n_2$ are the sample sizes.
• $\sqrt{s_p^2 \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}$ is the estimated standard error of the difference between the two means.

Under the null hypothesis, this t-statistic follows a t-distribution with degrees of freedom $df = n_1 + n_2 - 2$.

Test Statistic (Assuming Unequal Variances - Welch's t-Test)

If the assumption of equal variances is not met (which can be tested using methods like Levene's test), or if the sample sizes are very different, it is more appropriate to use Welch's t-test. The formula for the test statistic is slightly different:

where:

• $\bar{x}_1, \bar{x}_2$ are the sample means.
• $s_1^2, s_2^2$ are the sample variances.
• $n_1, n_2$ are the sample sizes.

The degrees of freedom for Welch's t-test are calculated using a more complex formula (the Welch-Satterthwaite equation), which usually results in a non-integer value. For practical purposes, software typically computes this value. In introductory contexts, the pooled t-test is often the focus.

Decision Rule

The decision rule for the two-sample t-test (pooled or Welch's) is the same as for the one-sample t-test, but using the appropriate degrees of freedom ($df = n_1 + n_2 - 2$ for pooled, or the calculated value for Welch's) and comparing the calculated t-statistic to the critical value(s) from the t-distribution or using the p-value.

---

Examples

Answer:

Given:

Hypotheses:

Solution:

---

Answer:

Given:

Hypotheses:

Solution: